Improving Speech Understanding through Integration of Prosody and Syntax

As automatic speech recognition technology improves and as it is applied to more difficult tasks, the need to integrate with syntactic, semantic, pragmatic and knowledge-based system components will increase. This paper covers the development of models which relate prosodic phrasing to syntactic structure. These models are useful for understanding the linguistic aspects of the prosody-syntax relationship and have potential for improving the performance of speech recognition systems. There are three important results from these models. Firstly, the use of a link grammar provides an effective syntactic framework for predicting prosody that is simpler and more accurate than conventional constituent analysis. Secondly, with the use of the link grammar framework, the left syntactic context is important for predicting prosodic structure while the right context is not significant. Thirdly, the models lead to a novel extension to the n-gram stochastic language model mechanism which incorporates prosodic constraints in a way that can be directly integrated with HMM-based speech recognition.